-
Notifications
You must be signed in to change notification settings - Fork 25.4k
Speed up (filtered) KNN queries for flat vector fields #130251
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
For dense vector fields using the `flat` index, we already know a brute-force search will be used—so there’s no need to go through the codec’s approximate KNN logic. This change skips that step and builds the brute-force query directly, making things faster and simpler. I tested this on a setup with **10 million random vectors**, each with **1596 dimensions** and **17,500 partitions**, using the `random_vector` track. The results: ### Performance Comparison | Metric | Before | After | Change | | ----------------- | --------- | ---------- | --------- | | **Throughput** | 221 ops/s | 2762 ops/s | 🟢 +1149% | | **Latency (p50)** | 29.2 ms | 1.6 ms | 🔻 -94.4% | | **Latency (p99)** | 81.6 ms | 3.5 ms | 🔻 -95.7% | Filtered KNN queries on flat vectors are now over 10x faster on my laptop!
Pinging @elastic/es-search-relevance (Team:Search Relevance) |
Hi @jimczi, I've created a changelog YAML for you. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am loving these numbers. Thank you for digging into this!
server/src/main/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapper.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/search/vectors/RescoreKnnVectorQuery.java
Outdated
Show resolved
Hide resolved
server/src/main/java/org/elasticsearch/index/mapper/vectors/DenseVectorFieldMapper.java
Outdated
Show resolved
Hide resolved
…e_force_knn_optim
I tweaked the brute-force nested version so that it always diversifies the child docs. It’s similar in spirit to what we do for HNSW, but much faster here since we can just walk through each parent’s block in order. I ran a comparison between
As you can see, the performance drops off pretty hard in the nested filtered case, lots of room for improvement. I think we should tackle that in a follow-up. From profiling, stuff like |
server/src/main/java/org/elasticsearch/search/vectors/DiversifyingParentBlockQuery.java
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
OK, if this passes given the new yaml tests, I think this is g2g!
Well it's broken ;) |
Ok that was an issue when the diversifying query hits NO_MORE_DOCS. |
…lastic#130263) PR elastic#130251 made me realize we were missing some important coverage. This adds nested vector query (and top level knn) tests for flat indices in our yaml tests.
For dense vector fields using the `flat` index, we already know a brute-force search will be used—so there’s no need to go through the codec’s approximate KNN logic. This change skips that step and builds the brute-force query directly, making things faster and simpler. I tested this on a setup with **10 million random vectors**, each with **1596 dimensions** and **17,500 partitions**, using the `random_vector` track. The results: ### Performance Comparison | Metric | Before | After | Change | | ----------------- | --------- | ---------- | --------- | | **Throughput** | 221 ops/s | 2762 ops/s | 🟢 +1149% | | **Latency (p50)** | 29.2 ms | 1.6 ms | 🔻 -94.4% | | **Latency (p99)** | 81.6 ms | 3.5 ms | 🔻 -95.7% | Filtered KNN queries on flat vectors are now over 10x faster on my laptop!
For dense vector fields using the
flat
index, we already know a brute-force search will be used, so there’s no need to go through the codec’s approximate KNN logic. This change skips that step and builds the brute-force query directly, making things faster and simpler.I tested this on a setup with 10 million random vectors, each with 1596 dimensions and 17,500 partitions, using the
random_vector
track. The results:Performance Comparison
Filtered KNN queries on flat vectors are now over 10x faster on my laptop!